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Abstract-  Models  of  attention  and  saccade  target  selection 
propose  that  within  the  brain  there  is  a  topographic  map  of  visual 
salience  that  selects,  through  a  winner-take-all  mechanism, 
locations  for  further  processing.  The  results  of  a  series  of  recent 
experiments  in  monkeys  performing  pop-out  visual  search  tasks 
suggest  that  the  frontal  eye  field  (FEF)  functions  as  a  map  of  visual 
salience.  FEF  is  located  at  the  interface  of  sensory  and  motor 
processing  and  participates  in  the  transformation  of  visual 
information  into  a  command  to  move  the  eyes.  Visually  responsive 
neurons  in  FEF  identify  conspicuous  objects  in  a  search  array 
regardless  of  the  feature  that  renders  conspicuousness. 
Furthermore,  selection  occurs  at  a  constant  interval  following 
search  array  presentation  and  is  dissociated  from  saccade 
production.  The  finding  of  a  visual  salience  map  in  FEF  validates 
models  of  visual  selection  and  can  serve  to  guide  future  empirical 
and  theoretical  investigations. 
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I.  Introduction 

Vision  is  a  dynamic  process.  It  is  dependent  on  rapid  eye 
movements  called  saccades  that  redirect  the  fovea,  the  high 
acuity  region  of  the  retina,  onto  informative  elements  in  the 
visual  scene.  Before  each  saccade  our  eyes  remain  stable  for 
about  200  milliseconds  while  visual  processing  takes  place. 
During  this  period  of  fixation,  our  brains  must  decide  what 
object  to  look  at  next  and  then  prepare  and  execute  the  saccade. 

Gaze  is  often  directed  to  conspicuous  objects  that  stand  out 
from  the  background.  Visual  conspicuousness  occurs  when  an 
object  has  a  unique  feature  (e.g.  color,  luminance,  orientation, 
motion,  size)  that  sets  it  apart  from  the  rest  of  the  image.  The 
term  “p°P'0Ut”  is  often  used  to  describe  this  capturing  of 
attention  through  a  bottom-up  selection  process.  Bottom-up 
refers  to  the  automatic  allocation  of  attention  based  exclusively 
on  the  properties  of  the  image.  In  contrast,  top-down  refers  to 
selection  based  on  cognitive  factors  such  as  the  goals  and 
knowledge  of  the  viewer  (reviewed  in  [1]). 

In  1985  Koch  and  Ullman  introduced  the  idea  of  a  salience 
map  to  accomplish  visual  selection  [2],  A  salience  map  is  an 
explicit  two-dimensional  map  that  encodes  the  salience  of 
objects  in  the  visual  environment.  Salience  can  be  derived 
from  both  bottom-up  and  top-down  factors.  A  winner-take-all 
competition  within  this  map  gives  rise  to  a  single  location  that 
corresponds  to  the  most  salient  object  that  can,  blit  need  not 
necessarily,  be  the  target  for  the  next  eye  movement  (reviewed 
in  [3]).  Most  models  of  visual  search  have  incorporated  the 
idea  of  a  salience  map  within  the  brain  to  guide  covert  attention 
[4],  [5],  [6],  [7],  and  overt  gaze  shifts  [8],  [9],  [10]. 

The  spatial  pattern  of  activity  across  the  salience  map  must 
be  able  to  localize  the  most  salient  object  regardless  of  what 


feature  or  combination  of  features  is  responsible  for  its  salience. 
Thus,  the  neural  substrate  of  the  salience  map  cannot  itself  be 
sensitive  to  specific  visual  features;  otherwise  activation  could 
be  attributed  to  the  specific  feature  rather  than  overall  salience. 
In  the  models  mentioned  above,  specific  features  of  the  image 
are  encoded  in  parallel  at  lower  levels  of  the  system  within 
multiple  feature  maps.  These  feature  maps  then  converge  onto 
one  map  of  visual  salience. 

Does  such  a  salience  map  actually  exist  in  the  brain? 
Recent  work  has  pointed  to  brain  structures  within  the 
visuomotor  system  that  may  encode  visual  salience.  Among 
these  are  the  pulvinar  [11],  the  posterior  parietal  cortex  (PP) 
[12],  [13],  [14],  the  superior  colliculus  [9],  [15],  and  the  frontal 
eye  field  (FEF)  [16],  [17]  (Fig.  1).  This  review  will  focus  on 
evidence  from  the  frontal  eye  field. 

II.  Frontal  eye  field 

The  frontal  eye  field  (FEF)  is  ideally  positioned  to 
contain  a  map  of  visual  salience  for  the  purpose  of  guiding  eye 
movements  (Fig.  1).  FEF  receives  topographically  organized 
converging  inputs  from  multiple  extrastriate  visual  cortical 
areas  [18].  Roughly  half  of  FEF  neurons  have  visual  responses 

[19] ,  [20],  [21].  The  output  of  FEF  is  from  a  population  of 
neurons,  called  movement  neurons,  that  are  located  in  layer  5 
and  discharge  specifically  before  and  during  eye  movements 

[20] ,  [22].  FEF  movement  neurons  send  an  eye  movement 
command  to  the  superior  colliculus  [23]  and  parts  of  the  neural 
circuit  in  the  brainstem  that  generates  saccades  [24]. 


Fig.  1 .  Simplified  schematic  of  anatomical  connections  within  the  visuomotor 
system  of  the  monkey.  Visual  information  from  multiple  visual  cortical  areas 
converges  onto  the  frontal  eye  field.  The  frontal  eye  field  sends  an  eye 
movement  command  to  the  superior  colliculus  and  brainstem  saccade 
generator,  which  sends  a  motor  signal  through  the  cranial  nerves  to  the  eye 
muscles. 
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Neurons  in  FEF  have  spatially  defined  response  fields  and 
FEF  is  topographically  organized.  The  ventrolateral  part  of 
FEF  represents  central  regions  of  visual  space  and  electrical 
stimulation  of  this  region  evokes  short  amplitude  saccades.  The 
dorsolateral  part  of  FEF  represents  peripheral  regions  of  visual 
space  and  electrical  stimulation  of  this  region  evokes  larger 
amplitude  saccades  [20],  [25]. 

The  visual  cortex  is  organized  into  functionally  specialized 
areas  that  contain  neurons  that  are  broadly  tuned  to  one  or  a  few 
feature  dimensions  [26],  [27].  The  processing  in  these  areas 
corresponds  conceptually  to  the  feature  maps  in  models  of 
visual  search  [2]-[9].  As  a  result  of  the  extensive  convergence 
from  extrastriate  visual  cortex,  the  visually  responsive  neurons 
in  FEF  do  not  exhibit  selectivity  for  stimulus  features  [28], 
[29],  but  instead  exhibit  activation  that  represents  visual 
salience.  The  remainder  of  this  paper  will  review  recent  work 
that  supports  this  view. 

HI.  Saccade  target  selection 
A.  Target  selection  during  visual  search 

The  visual  search  paradigm  has  been  used  extensively  to 
investigate  visual  selection  and  attention  [4],  [30].  In  a  visual 
search  task,  multiple  stimuli  are  presented,  and  from  among 
them  a  target  is  discriminated.  To  investigate  how  the  brain 


Fig.  2.  Visual  search  tasks.  Three  different  pop  out  search  tasks  were  used. 
The  tasks  were  run  in  separate  blocks  of  trials.  Monkeys  began  each  trial  by 
fixating  the  central  spot.  The  sequence  of  each  task  is  shown  on  the  left.  Eye 
position  traces  are  shown  on  the  right  indicating  the  monkey’s  behavior  during 
each  task.  Target  location  was  randomized  from  trial  to  trial.  A.  Go  search 
task.  A  search  array  appeared  and  the  monkey  was  rewarded  for  making  a 
saccade  to  the  oddball.  B.  Nogo  search  task.  Monkeys  were  rewarded  for 
maintaining  fixation  on  the  central  spot  for  the  entire  trial.  Monkeys  did  not 
indicate  location  of  target  in  this  task.  C.  Search-step  task.  Most  trials  were 
the  same  as  in  go  search  (A).  On  the  remaining  trials  the  target  swapped 
positions  with  a  distractor  after  a  short  delay  called  the  target  step  delay  and 
monkeys  were  rewarded  for  shifting  gaze  to  the  new  target.  On  search-step 
trials  monkeys  either  compensated  for  the  target  step  or  failed  to  compensate. 
Noncompensated  saccades  were  typically  followed  by  a  second  corrective 
saccade. 


selects  targets  for  visually  guided  saccades,  recordings  of 
neural  activity  were  made  in  the  FEF  of  monkeys  trained  to 
shift  gaze  to  an  oddball  target  stimulus  among  identical 
distractor  stimuli  (e.g.  red  target  among  green  distractors)  (Fig. 
2).  This  task  is  an  example  of  pop-out  search,  and  therefore  is 
controlled  by  a  bottom-up  selection  process. 

Initially,  saccade  target  selection  in  FEF  was  investigated 
using  a  go  search  task  (Fig.  2A)  [28],  [31].  It  was  found  that 
most  FEF  visually  responsive  neurons  responded  initially 
indiscriminately  to  the  target  or  the  distractor  of  the  search 
array  in  their  receptive  field.  However,  before  saccades  were 
generated,  a  discrimination  process  took  place  by  which  most 
visually  responsive  neurons  ultimately  signaled  the  location  of 
the  oddball  stimulus  (Fig.  3  A).  Thus,  the  activity  of  FEF  visual 
neurons  does  not  represent  specific  features  of  objects,  but 
rather  reflects  the  evolution  of  the  visual  selection  or  saccade 
preparation  processes. 

B.  Timecourse  of  target  selection 

Having  found  that  it  was  possible  to  observe  saccade  target 
selection  in  the  activity  of  FEF  visual  neurons,  an  analysis  was 
carried  out  to  determine  when  the  selection  was  accomplished 
and  the  time  of  target  discrimination  was  related  to  when  the 
saccade  was  made  [32].  It  was  found  that  FEF  visual  neurons 

Saccades 


Time  from  search  array  presentation  (ms) 

Fig.  3.  Visual  selection  of  a  conspicuous  target.  The  neural  activity  of  a 
single  FEF  visual  neuron  is  shown  following  presentation  of  a  pop-out  search 
array  during  (A)  GO  search  and  (B)  NOGO  search.  Each  plot  shows  the 
activation  when  the  oddball  stimulus  appeared  in  the  receptive  field  (RF)  (thick 
line)  and  when  distractors  appeared  in  the  receptive  field  (thin  line).  The  trials 
are  aligned  on  the  time  of  search  array  presentation.  A.  The  time  course  of 
activation  during  a  block  of  GO  search  trials.  The  activation  during  subsets  of 
trials  in  which  reaction  times  (RT)  were  short  or  long  are  shown  separately. 
The  plots  of  neural  activity  end  at  the  mean  reaction  time  for  each  group.  The 
ranges  of  reaction  times  for  the  short  and  long  trials  are  indicated  across  the  top. 
B.  The  time  course  of  activity  during  a  block  of  NOGO  trials.  The  times  of 
target  discrimination  (solid  arrows)  were  approximately  the  same  in  all  three 
subsets  of  trials  showing  dissociation  between  the  visual  selection  of  a  stimulus 
in  FEF  and  the  production  of  saccades  (modified  from  [15]). 
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discriminate  the  target  from  a  distractor  in  a  pop-out  search 
array  at  a  fairly  constant  interval  after  search  array  presentation 
(Fig.  3A).  In  other  words  the  time  at  which  the  target  of  the 
saccade  was  identified  did  not  predict  when  the  eyes  moved. 
This  finding  suggests  that  the  target  selection  process  in  FEF  is 
more  closely  linked  to  visual  processing  than  to  saccade 
production. 

III.  Dissociation  of  visual  selection  from  saccades 

A.  Visual  selection  without  saccades 

To  examine  further  the  dissociation  of  visual  selection  in 
FEF  from  saccade  production,  Thompson  et  al.  [33]  tested  the 
hypothesis  that  the  selection  observed  in  FEF  does  not  require 
saccade  planning  and  execution.  FEF  activity  was  recorded 
while  monkeys  were  instructed  to  maintain  fixation  during 
presentation  of  the  pop-out  search  array  (Fig.  2B).  Although  no 
saccade  was  made  to  the  oddball,  FEF  neurons  still 
discriminated  the  oddball  from  distractors  at  the  same  time  and 
to  the  same  degree  as  when  a  gaze  shift  was  produced  (Fig. 
3B).  Thus,  the  visual  selection  observed  in  FEF  identifies 
potential  targets  for  saccades,  blit  does  not  require  the 
generation  of  saccades. 

B.  Visual  selection  with  competing  saccade  targets 

The  nogo  search  experiment  [33]  showed  that  the  visual 
selection  process  in  FEF  is  not  dependent  on  the  production  of 
a  saccade.  However,  because  monkeys  had  been  trained  to 
make  saccades  to  the  target,  it  could  be  argued  that  the 
observed  selection  reflected  some  kind  of  latent  saccade 
programming.  The  search  step  task  (Fig.  2C)  was  designed  to 
test  this  possibility  and  to  ultimately  determine  if  the  selective 
activity  in  FEF  reflects  perceptual  or  motor  processes  [34]. 

In  the  search-step  task  most  trials  were  identical  to  go 
search,  but  on  about  a  third  of  the  trials  the  target  unexpectedly 
switched  places  with  a  distractor.  The  monkey’s  task  was  to 
shift  gaze  to  the  new  target  location  to  receive  a  juice  reward. 
The  timing  of  the  target  step  was  adjusted  so  that  the  monkeys 
were  able  to  compensate  for  the  target  step  on  about  half  the 
trials.  On  the  remaining  trials  the  monkeys  did  not  compensate 
for  the  target  switch  and  initially  looked  at  the  original  target 
location  and  were  not  rewarded.  The  question  to  be  answered 
was:  Does  the  selective  activation  reflect  the  location  of  the 
pop-out  target  or  does  the  selective  activation  predict  the 
monkey’s  impending  eye  movement?  The  answer  is  that  the 
selection  reflects  the  visual  stimulus,  not  the  monkey’s 
behavior. 

Figure  4  shows  the  typical  results.  The  early  phase  of  the 
response  during  search-step  trials  was  identical  to  the  response 
elicited  by  a  distractor  during  go  search  trials;  the  early 
nonselective  visual  activity  was  followed  by  suppression.  This 
is  because  the  stimulus  conditions  were  identical  prior  to  the 
target  step.  The  activation  grew  markedly  following  the  target 
step.  Furthermore,  the  growth  of  activity  on  trials  in  which 
monkeys  compensated  with  a  saccade  to  the  final  target  located 


Fig.  4.  Frontal  eye  field  visual  neuron.  A.  Activation  when  the  target  (filled) 
or  distractors  (open)  fell  in  the  receptive  field.  Following  the  initial  100  msec 
of  activation  that  did  not  discriminate  target  from  distractor,  the  activity  was 
modulated  strongly.  The  response  to  the  distractor  was  suppressed,  and  the 
response  to  the  target  grew.  B.  Averaged  activity  on  compensated  (thick 
black)  and  non-compensated  (dotted)  target-step  trials  when  the  distractor  in  the 
receptive  field  unexpectedly  became  the  target  compared  with  activity  on  no¬ 
step  trials  when  distractors  remained  in  the  receptive  field  (thin  black).  In  both 
compensated  and  noncompensated  trials  the  neuron  responded  equally  strongly 
to  the  unexpected  appearance  of  the  target  in  the  receptive  field.  The  fact  that 
the  activity  before  the  mean  noncompensated  saccadic  reaction  time  (vertical 
dashed  line)  was  indistinguishable  for  the  search-step  trials  with  opposing 
saccade  directions  means  that  the  activity  of  this  neuron  could  not  be  involved 
in  saccade  production.  Only  presaccadic  spikes  were  used  in  constructing  the 
response. 

in  the  receptive  field  was  identical  to  the  growth  of  activity  on 
trials  in  which  monkeys  failed  to  compensate  and  produced  a 
saccade  to  the  original  target  located  outside  the  receptive  field. 
In  other  words,  the  activity  of  this  neuron  represented 
accurately  the  new  location  of  the  target  regardless  of  whether 
compensated  or  noncompensated  saccades  were  produced. 
This  is  strong  evidence  that  this  selection  process  is  distinct 
from  immediate  saccade  production. 

These  findings  suggest  that  FEF  may  play  a  role  in  covert 
orienting  of  visual  attention  as  well  as  overt  saccade  generation. 
This  conclusion  is  supported  by  psychophysical  studies 
showing  that  attention  is  allocated  automatically  to  the  pop-out 
oddball  in  a  search  array  [35]  and  by  recent  brain  imaging 
studies  showing  that  a  region  including  FEF  in  human  frontal 
cortex  is  activated  in  association  with  both  attention  and 
saccade  tasks  [36],  [37]. 

IV.  Conclusion 

Models  of  visual  attention  and  saccade  target  selection  posit 
the  existence  of  a  map  of  visual  salience  in  the  brain  that 
specifies  locations  for  further  processing  based  on  the 
properties  of  the  image.  The  stage  of  processing  represented  by 
the  salience  map  is  distinct  from  motor  preparation  processes 
because  it  may  or  may  not  lead  to  an  overt  orienting  response 
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such  as  a  saccade.  The  characteristics  of  the  visual  selection 
observed  in  the  population  of  FEF  visually  responsive  neurons 
are  consistent  with  it  being  a  salience  map.  FEF  visual  neurons 
exhibit  a  selective  response  for  conspicuous  stimuli  at  a  time 
that  is  linked  to  the  presentation  of  the  visual  stimulus.  Also, 
visual  selection  occurs  regardless  eye  movement  behavior.  It 
seems  clear  that  a  functional  salience  map  exists  in  FEF,  and 
further  work  will  determine  how  this  map  may  be  distributed 
across  other  visuomotor  areas  such  as  the  posterior  parietal 
cortex  and  the  superior  colliculus.  Nevertheless,  the  idea  of  a 
salience  map  appears  to  be  a  physiologically  sound  theoretical 
construct  that  will  be  useful  in  guiding  future  theoretical  and 
empirical  investigations. 
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